02:59
2026-06-20
dev.to
large-language-models
Load late, load little: just-in-time context for conversation history
A developer introduced a just-in-time context system for conversation history that keeps a compact hot index of summaries and embeddings in the window while fetching full turns on demand, reducing cos…